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1. Introduction 

The notion of an (adaptive) landscape has proved to be a valuable concept in theoretical 
investigations of evolutionary change, combinatorial optimization, and the physics of 
disordered systems. From the mathematical point of view, a landscape consists of three 
ingredients: (i) a set V of "configurations" which we shall assume to be finite but very 
large, (ii) a cost or fitness function / : V — > R that evaluates the configurations, 
and (iii) some sort of additional geometrical, topological, or algebraic structure X 
on V that allows us to define notions of closeness, similarity, or dissimilarity among 
the configurations [1, 2, 3]. In the simplest case, X is an adjacency relation. In this 
contribution we shall consider systems consisting of N Ising spins and we shall assume 
that two spin configurations x and x' are adjacent when they differ in the orientation 
of a single spin, x' k = —Xk- We say that x G V is a local minimum of the landscape / if 
f(x) < f(y) for all adjacent configurations (neighbors) y of x. The use of < instead of 
< is conventional [4, 5]; it does not make a significant difference for spin glass models. 
Local maxima are defined analogously. The number Af of local optima in a landscape 
may serve as a measure for the landscape's ruggedness [6]. 

Alternatively, ruggedness can be measured by means of correlation functions such 
as r(s), defined as the autocorrelation function of the "time series" f(x t ) sampled along 
an unbiased random walk of s steps on the configuration space [3]. Explicitly, r(s) can 
be represented as a quadratic form 

r( a ) = (/,(D- 1 A)V)/(/,/) (1) 
where A is the adjacency matrix of the configuration space (V,X), D is the diagonal 
matrix of its vertex degrees, i.e., D xx is the number of neighbors of each configuration, 
f(x) = f(x) — /, and / = | V" | 1 f(x) [1]. It is not hard to verify that r(s) is an 
exponential function if and only if / is an eigenvector of A. Such landscapes have been 
termed elementary. The importance of elementary landscapes derives in part from the 
fact that all Ising spin models 

1~L (x) ^ ^ J'i\i2...i p %ii%i2 • • ■ "^ip (2) 

(ii<i2<. ••<%,) 

with a fixed interaction order p are elementary, with an eigenvalue N — 2p that depends 
only on p and not on the details of the index set (ii < i<i < . . . < i p ) of non-vanishing 
spin interactions. In the case that the couplings J^...^ are statistically independent, 
Gaussian distributed random variables, Eq.(2) defines Derrida's p-spin Hamiltonian [7], 
which for p = 2 reduces to the well-known SK model [8] . 

The information contained in r(s) is conveniently further condensed into the 
correlation length £ = X]fc^=o r ( s )- ^ or elementary Ising spin models we obtain 
immediately I = N/ (2p), see e.g. [9, 10]. It would appear that M and i are two sides of 
the same coin, and hence we would expect a close connection between the two measures. 
Indeed, for Derrida's p-spin Hamiltonian the expected number of local optima (Af) scales 
like exp (aN) with a increasing from 0.199 for p = 2 to In 2 0.692 for p — > oo [11]. 
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This increase of a, and hence of the number of local optima, matches the decrease of 
the correlation length £ with increasing p . 

Although for random landscapes (Hamiltonians with disorder) it is often desirable 
to determine (In A/"), in most cases one has (In Af) = In (A/") (a notable exception is 
the linear spin chain [12]). The reason for this equality is that the overlap between 
two randomly chosen metastable states vanishes with probability one and so the replica 
approach needed to evaluate the average of In Af reduces to the annealed approximation, 
which takes the average directly on Af [13]. Of course, this is no longer true if one 
considers specific classes of metastable states (e.g. those possessing a given energy 
density) or if one adds an external magnetic field to the Hamiltonian (2) [14]. 

In [1, 2] the notion of an isotropic random landscape was introduced as a 
"statistically symmetric model", that 1S ; clS db random landscape with a covariance 
matrix C xy = (f(x)f(y)) — (f(x)) (f(y)) that shares the symmetries of the underlying 
configuration space. An Ising spin glass is isotropic if and only if all interaction 
coefficients J^...^ are uncorrelated with mean ( J^...^) = 0, and those J^...^ that belong 
to a common interaction order have the same variance, ^J? i^j = °~ 2 (p) [2]- In other 
words, the elementary isotropic Ising models are exactly the p-spin Hamiltonians with 
infinite-range interactions. It is argued at length in [15, 16] that isotropy can be 
interpreted as a maximum entropy condition. The properties of the infinite-range p- 
spin Hamiltonians are obviously determined exclusively by the number of spins N and 
the interaction order p or, equivalently, by the correlation length t. 

Short-range spin models, in which only a small fraction of all (^) possible p-ary 
spin interactions contribute to the Hamiltonian, i.e., (Jf li2 ip ) = for most p-ary spin 
patterns {i\ < %2 < ■ ■ ■ < i p ), deviate significantly from isotropy. For the SK model 
{p — 2) a slightly larger number of local optima has been found [17, 18] than for the 
infinite-range case [19]. The deviation is proportional to 1/z, where z is the number of 
the nearest-neighboring points in a hypercubic lattice of dimension d = z/2. 

In this contribution we show that an analogous effect is at work in higher order spin 
glasses. In addition, numerical simulations for small values of connectivities do not show 
a substantial dependence on the patterns of interacting spins. The rest of this paper 
is organized in the following way. In section 2 we generalize the variational method 
of Tanaka and Edwards [17] to estimate the 1/z corrections to the expected number 
of metastable states of Derrida's p-spin model. The results of numerical simulations 
of fourth-order spin glasses in two- and three-dimensional cubic lattices with small 
connectivities are discussed in section 3. In section 4 we investigate the metastable states 
of a highly frustrated one-dimensional spin model without explicit disorder, namely, the 
low autocorrelated binary string problem [20]. Finally, in section 5 we present some 
concluding remarks. 
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2. Short-Range p-spin Models 



The coupling strengths J^...^ in Eq.(2) are modeled as statistically independent random 
variables with the Gaussian distribution 

, 2 



*P {Jiii-z—ip) 



< z p- 



Tip\ 



cxp 



p\ 



9[(ii <i 2 < ... < i P )\ (3) 



where < %2 < ■ ■ ■ < i P )] = 1 if (*i < . . . < i p ) is a valid interaction pattern and 

otherwise. Since in practice it is not feasible to consider a fixed interaction pattern, 
we consider all interaction patterns with given fixed coordination number z. In other 
words, we sum over all ways of choosing the p — 1 spins among the z allowed ones. 

As an immediate consequence of Eq.(2) the energy cost of flipping the spin Xj is 
5H = 2\j, where 

= ^ ] Jji2—i P x j x ii ■ ■ • x i P (4) 
(i 2 <...<i p ) 

is the stability of X; L . Hence any state x that satisfies 

\ > Vi (5) 

is a local minimum of the random landscape defined in Eq.(2). Thus the number of 
local minima can be written as 



J\f = Tr x Y[ dXjSlXj- ^2 J in-ip x 3 

j J ° \ (ia<...«p) 



Xj 2 - - - %i p 



(6) 



where Ti x denotes the summation over the 2 N spin configurations and 5 (x) is the Dirac 
delta function. 

In the following we will calculate analytically the expected number of metastable 
states (A/") in the limit of large N and z with N ^> z. Here (. . .) stands for an average 
over the coupling strengths in all possible interaction patterns with fixed coordination 
number z. Using the integral representation of the delta function, the average over the 
couplings as well as the summation over the spin configurations can be easily performed 
[14], yielding 



(ao = n r d\ r ^ exP ^0 

\.J0 J -00 K 



x exp 



p\ 



4zp- 



(7) 

(h<i2<---<i P ) 

Clearly, the expansion of the quadratic term in the argument of the exponential 
function will lead to interactions terms of second order in the auxiliary fields 0j. More 
precisely, using 

#t E *£ = ^*(*-l)-(*-l>+2>E# 



(ii<...<ip) 



2z 



(p-l)(p-2) + 0(z- 2 ) 



(8) 
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and 



pi 

zP -i 



(ii<...<i p ) 



(z-l)(z-2)...(z-p + 2)J2M j 

(i<3) 



(»<j) 



(9) 



we write Eq.(7) as 

(A0 = 

where 



j j d0i -D (0^) exp 



2z 



-E 



(*<i) 



1 + 



p(p-l)(p-2) 

82 



+ (^ 2 ) 



dA 



7T 



e -iA0i-p</>?/4 



(10) 



is the field's weight function. As mentioned before, Eqns.(8) and (9) follow from the 
sum over all possible interaction patterns with coordination number z. More precisely, 
for each site i there are precisely z sites k such that (i, k, . . . , i p ) is a valid interaction 
pattern for some choice of i 3 ,...,i p . This notion of site connectivity is independent 
of the lattice dimensionality; In fact, it is not necessary to assume that the sites are 
arranged on a lattice at all: Eq.(10) remains valid as long as z is site independent. 
Hence we will refer to z simply as the connectivity of our model. 

To proceed further we must evaluate the integrals in Eq.(10) taking the care to 
collect all terms of first order in \jz. In particular, note that >~2(i<j) 1 = Nz/2. This 
can be achieved through an ingenious variational method introduced by Tanaka and 
Edwards in their analysis of the case p = 2 [17]. The idea is to add an auxiliary 
single-particle term to the effective Hamiltonian of Eq.(10) which is then rewritten as 



(AO = [* (01 



N 



cxp 



2z 



(i<j) 



(12) 



where the average (...)* is defined by 

/ ft d& £(&)(...) exp (if? 1 /* £.0.) 



(■■■>t = 



fUi^iD (&) exp (itpi/2 



(13) 



and 



/oo 
d0D(0) exp (itp 1/2 <p) 
-oo 



1 t 



erfc (-*) + — = (p -l)ip- 2) e"* + O (z~ 2 ) 

Z V 4:71 



For example, we find 
,V2 



i p 

-p 



'kit 



J klt 



mm) 



(14) 

(15) 
(16) 
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for k = 1, . . . , N. Here t is a variational parameter which will be determined so as to 
maximize (A/") . At this point we can note that the 1/z expansion is valid provided that 
z > p 2 . 

As usual, the average in Eq.(12) can be evaluated through the cumulant expansion. 
In particular, we assume that only the first cumulant contains terms of zeroth order in 
1/z and it is solely these terms that determine the value t — t m that maximizes (A/) [17]. 
Of course, this assumption must be verified a posteriori through the explicit calculation 
of the higher order cumulants. Evaluating the first cumulant we find that t m maximizes 
the expression 

ln$(t) - ^^12(0^ - \tp^{<t> k ) t (17) 
and so it is given by the solution of the equation 

t m = l(p-l)ip 1/2 {<!>k)t m . (18) 

The usefulness of the variational approach becomes transparent only when we 
rewrite Eq.(12) replacing t by t m and rearranging the terms: 



(A/") = exp 



Aln$(U + A^-iV>? 



x (exp(S)), (19) 



where 



s = -^7^ E ( & ~ ) ( - <^ ) • ( 2 °) 

(i<j) 

Since (S) t = 0, it is now straightforward to evaluate the higher order cumulants in the 
cumulant expansion 

(exp(S)> tm =exp^l(S»> w j . (21) 

In particular, we find 

=<>. ^ (22) 

<=\, ; c=^^<(#-w«,j 2 }; m , (23) 

( S3 } t ^=-" lj j^ 1 <(^-(«.J 3 >L- (24) 
More generally, we can easily show that (S n ) tm . c is of order z x ~ n . Therefore, to keep 
terms up to 1/z we need to consider only the second cumulant in Eq.(21). Moreover, 
the leading terms of the 1/z expansion (i.e., the terms of zeroth order in 1/z) are those 
outside the average symbol in Eq.(19). Clearly, these results remain unchanged by 
the fact that for p > 2 the averages ((/>%) t m also contribute to the 1/z corrections, see 
Eqns.(14-16). Hence, 

lln(A")= ln$(t m ) + ^^(0 fc )L 

+ ((^-WLjVolO- (25) 
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Table 1. Values of ao and a\ obtained from Eqns.(26), (28), and (29). 



V 


2 


3 


4 


5 


6 


10 


a 


0.1992 


0.2956 


0.3552 


0.3965 


0.4273 


0.5001 




0.0656 


0.3393 


0.7789 


1.3411 


1.9985 


5.2894 



erfc (-t m ) ' 

Inserting this result in Eqns.(14), (15) and (16), we rewrite Eq.(25) as 



a + -e*i + O (z 2 ) 



with 



The next step is to separate the contributions of zeroth and first order in 1/z. To 
achieve this we note that t m is determined only by the zeroth order term in the 1/z 
expansion so that Eq.(18) reduces to 

p - 1 exp (-t 2 J ^ 

(27) 
(28) 

.(p-iy ' v j ' (29) 

In Table 1 we summarize the values of «o an d a i obtained from a numerical solution 
of Eq.(26). For p — > oo we find t m ~ \^lnp and so ao — > In 2 while a\ diverges like pin p. 
Of course, ot\j z <C 1 since, as mentioned before, the 1/z expansion is consistent only if 
z ^> p 2 . We note that our results for p = 2 are in agreement with those of ref. [17]; the 
values of ao agree with the TAP solution [11], and with a numerical survey [21]. 



lim — In (AO = a 



a = In [erfc (—t r , 



P 



ai 



1.2 



+ 



P 



3. Simulations 

The analytical results in section 2 are valid for very large connectivities, z ^> p 2 , and for 
superimposed interaction patterns. Actually, the later restriction amounts to consider 
all interactions patterns as equivalent, in the sense that they have on the average the 
same number of local minima. In order to investigate the effect of distinct interaction 
patterns we consider in this section a variety of 4-spin models with small connectivities. 

For the sake of computational simplicity we consider only translation invariant 
interaction patterns P on two- and three-dimensional cubic lattices with periodic 
boundary conditions. We identify each spin by its d- dimensional lattice coordinates 
and take all indices modulo the lattice sizes mi through m^. In two dimensions, for 
instance, translation invariance means j±; i2,j2', h, jz\ U, j*) G P if and only if 
(h + P: ji + T,i2+ P, 32 + q; h + P, jz + T, U + P, U + q) £ P for all integers p mod m x and 
q mod m2- The Hamiltonian (2) becomes 

7~i(x) — ^ ^ Jhji\i2j2\h3rMjA X hjl X i232 X hj-i X UjA (30) 

(*lil,*2i2,«3j3,*4i4)eP 
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Figure 1. Interaction patterns of various short-range 4-spin models. For each grid 
position (indicated by the open circle) a non-zero contribution to the energy depends 
on the four spins indicated by the black circles. 

in this example. A variety of interaction patters on a square lattice, compiled in Figure 1 
and named according to the number and type of combinations (patterns) of spins that 
contribute to the Hamiltonian (30), have been used. In addition, we have considered 
three patterns on a cubic lattice: (i) The "positive orthant", coupling each spin Xij^ 
with its lattice neighbors along Xj+i^fc, Xij+i^ and Xij^+i, (ii) the pattern S restricted 
to a fixed plane, i.e., coupled with Xij+i^, Xij^+i, an d ^i,j+i,fc+i, and (iii) the 

couplings with the neighbors in all eight orthants. 

The relevant quantity for comparing the simulations with the analytical theory 
is the connectivity z of a lattice site j, that is, the number of spins k ^ j such that 
Jjki 3 i 4 7^ for some is and i±. The value of z depends of course strongly on the interaction 
pattern, see Table 2. Note that z is independent of the lattice dimensionality and of the 
number X of non-zero interaction strengths per site. 

Numerical simulations were performed by sampling up to 10 8 spin configurations 
from at least 10 5 different instances (random assignments of the coupling constants) for 
each model and values of N = Ylt=i m k between 8 and 60. The standard deviations 
listed in Table 2 are statistical errors from fitting the curve In (N) versus N to a straight 
line. With the exception of the cubic models, where we have only few data points, and 
the models qlq, q2q, which show quite strong finite size effects, the correlation coefficient 
of the linear regression is g > 0.9998. 

By comparison with the numerical estimates for the infinite-range 4-spin glass we 
suspect that systematic errors might be slightly larger than the statistical errors. A 
conservative estimate appears to be an over all accuracy of at least ±0.012. We also 
note that replacing the Gaussian distribution of J^...^ by a uniform distribution with 
mean apparently does not significantly influence the number of metastable states. 

The results of Table 2 show a clear tendency of decrease of a with increasing 
connectivities z, as expected from the calculations of section 2. Moreover, for fixed z and 
within the estimated accuracy, the values of a seem to be independent of the geometry 
of the interaction patterns. This interesting result indicates that the properties of short- 
range p-spin Hamiltonians may be fully characterized by a few macroscopic parameters, 
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Table 2. Estimated values of a for various short-range 4-spin models. 
For each pattern we list the number X of non-zero interaction coefficients per spin, the 
site connectivity z, the best numerical estimate for a and standard deviation estimated 
from the linear regression. 
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0.0014 
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0.0026 
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[21] 




TAP 
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0(N 2 ) 
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0.4634 


0.0011 


LABSP 


periodic 


0{N 2 ) 


0{N) 


0.4713 


0.0020 



namely, N, p and z. Indeed, the error between our theoretical predictions for p = 4 
(see Table 1) and the numerical data for the two-dimensional model with the largest 
connectivity, qAq (z = 24), is only 3.2%. (Comparison of the simulation data with 
the TAP solution for z — > 00 yields an error of 5.4%.) Since the estimated systematic 
error is 1.2% the agreement between theory and simulations is quite good and provides 
additional evidence of the minor role played by the geometry of the interaction patterns 
on the landscape properties of p-spin models. 

4. The Low Autocorrelated Binary String Problem 

The Low Autocorrelated Binary String Problem (LABSP) [20, 22] consists of finding 
binary strings x of length N over the alphabet {±1} with low aperiodic off-peak 
autocorrelation Rk(x) = Yli=[ k x i x %+k for all lags k. These strings have technical 
applications such as the synchronization in digital communication systems and the 
modulation of radar pulses. In the periodic variant of this model, one considers 
Rk{x) = J2iLi x i x i+k with indices taken mod A. The quality of a string x is measured 
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by the fitness function 

JV-l 

f{x) = Y,Mx?- (31) 

k=l 

In most of the literature on the LABSP the merit factor F(x) = N 2 / (2f(x)) is used (see 
[20]): using / instead is more convenient for explicit computations. 

Recently there has been much interest in frustrated models without explicit 
disorder. The LABSP and related bit-string problems have served as model systems 
for this avenue of research [23, 24, 25]. These investigations have shown that LABSP 
has a golf-course type landscape structure, which explains the fact that it has been 
identified as a particularly hard optimization problem for heuristic algorithms such as 
simulated annealing (see [26, 27] and the references therein). In this section we show 
that LABSP has by far more local optima than one would expect from its correlation 
length or interaction order. 

We use the fact that every function on the hypercube {+1, — 1}^ can be written as 
a linear combination of the p-spin functions £i u i 2 ,...,i p (x) = x^x^x^ . . .Xi p to translate 
LABSP into explicit spin glass form. Explicitly, we have for the aperiodic model [1] 

rf-l- 1 A r -2fc N-1N-1 

f(x) = a + ^ Y 2£ i,i+k( X ) + Y Y Y e 

i,i+k,j,j+k( x ), (32) 

k=l i=l k=l i=l jy^i—k,i,i+k 

where a$ is a constant factor which does not depend on N. A similar expression 
can be derived for the periodic model. There are roughly iV 2 /4 non-zero second 
order contributions and on the order of N 3 non-zero fourth order contributions. 
Thus the relative weights (amplitudes) of the 2-spin and the 4-spin contributions are 
B 2 = 0(1/N) and B 4 = 1 - 0(1/ N), respectively. For the general definition of B p 
see [1]. The landscape of LABSP thus consists of a (dominant) 4-spin Hamiltonian 
plus an asymptotically negligible quadratic component. The correlation length £ is 
therefore given by i — N/8 + 0(1), which is in excellent agreement with the estimate 
I m 0.123 • iV — 0.983 from numerical simulations [28]. We note that the generic 4-spin 
landscape is Derrida's 4-spin Hamiltonian [7] which is a linear combination of all (^) 
distinct 4-spin functions, while Eq.(32), on the other hand, only contains 0(N 3 ) non- 
vanishing 4-spin contributions. The landscape of the LABSP thus corresponds to a dilute 
4-spin glass. 

Table 2 summarizes a numerical survey of the local minima of the LABSP. For N < 70 
we have generated up to 10 8 spin configurations at random and checked whether they 
are local minima. As expected, the number of spin configurations that are local minima 
increases exponentially as exp(o;iV). The non-exponential pre-factor depends strongly 
on iVmod4: separate estimates of a from data for iVmod 4 = 0, 1, 2, 3 show deviations 
of up to ±0.005 within each of the variants of the LABSP, while the standard deviations 
from linear regression within each data set is almost an order magnitude smaller. We 
conclude that the discrepancy of 0.008 between the best estimates for aperiodic and a open , 
is not significant, since it is about the same size as the iVmod 4 dependence. 



Metastability in Short Range Spin Models 



11 



5. Conclusion 

Metastable states are an important aspect of a landscape's ruggedness. In this 
contribution we have shown that anisotropics, that is, deviations from a maximum 
entropy condition, significantly influence the frequency of metastable states, even when 
the interaction order p of the spin glass model, and therefore the correlation length of 
the resulting landscape, is kept constant. 

The correlation length £ was introduced as an easy-to-compute measure of 
ruggedness. Later on, it turned out that it has desirable algebraic properties, in 
particular in the context of Fourier transformation theory for fitness landscapes [1]. 
The inability of £ to reflect the ^-dependence of (A/") is certainly a weakness of this 
measure, in particular when ruggedness measures are used to predict the performance 
of optimization heuristics. 

The anisotropy of short-range spin glass models is the consequence of a large 
number of vanishing coupling constants compared with the corresponding infinite- 
range p-spin model. Following earlier work by Tanaka and Edwards [17] we have 
determined the influence of a finite connectivity z on the mean number (A/) of metastable 
states. Our result shows that, to first order in 1/z, the number of metastable states 
increases, i.e., short-range spin glasses are in general more rugged than their infinite- 
range counterparts. Moreover, the finite z effect becomes much more pronounced as p 
increases since the coefficient of 1/z diverges like pin p. Of course, large anisotropies 
due to very small lattice connectivities cannot be dealt with by our approach, which is 
also based on the assumption that all interaction patterns with fixed connectivity are 
equivalent. However, numerical simulations of several fourth-order spin glasses show 
that in these cases (Af) seems to depend only on the lattice connectivity and not on the 
detailed geometry of the spin interactions. 

The low-autocorrelated binary string problem may be regarded as a strongly 
anisotropic 4-spin model, where the interaction strength can assume only the values 
and 1. This model is a particularly hard optimization problem [20, 27]. In this 
contribution we have shown that it exhibits a number of local optima that is by far 
larger than expected for a 4-spin model, even taking into account the reduced number 
of non-zero coefficients. 

Our results indicate that a large class of generic short-range p-spin Hamiltonians 
may be fully characterized by a few macroscopic parameters, namely, N, p and z. 
Particular anisotropic (non-generic) constructions, on the other hand, need not conform 
to this picture, as the example of the LABSP shows. 
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